#14 Self-Distillation: Towards Efficient and Compact Neural Networks
Date:
Instead of transferring knowledge from a teacher model to a different student model, self-distillation does it in the same model! This approach leads to faster inference and smaller models! Link for the video